1. Technical Field
This present disclosure provides a unique design solution using a direct current (DC) uninterruptible power supply (UPS) and server power supply with DC input voltage for the design of high-efficiency, cost-effective modular data centers.
The present disclosure also provides unique design solutions for high density modular data centers by using either: (i) an alternating current (AC) uninterruptible power supply (UPS) in energy saver (ES) mode; or (ii) a direct current (DC) UPS in conjunction with a server power supply with a DC input voltage.
2. Background of Related Art
There is a large demand for efficient Data Centers to store large amounts of data due to the emergence of Web 2.0-enabled businesses such as financial, e-commerce, pharmaceutical, or multi-media businesses. Indeed, information technology (IT) growth is outstripping Moore's law, which is a rule of thumb where the number of transistors that can inexpensively be placed in an integrated circuit (IC) doubles approximately every two years. However, in the past many years, major energy efficiency improvements and technological innovation has not been achieved for both electrical and mechanical infrastructure of the data center industry, even as computing hardware and software has become much more efficient. Thus, efficient computing hardware and software sits on an inefficient electrical and mechanical infrastructure. This inefficient infrastructure represents significant capital expenditure (CAPEX) and operational expenditure (OPEX) cost problems for businesses. Data centers and businesses with mission critical data storage requirements need a high-efficient and reliable infrastructure systems to reduce overall total cost of ownership (TCO).
By some estimates, the demand for data storage doubles approximately every 18 months, which results in an annual growth rate of approximately 150% for the next 5 years. This increase in demand for data storage is linearly proportional to the increase in the amount of data being processed, which is being driven by the increase in mobile data traffic and data processing of large, medium, and small business operations.
Modularity and flexibility are key elements in allowing for a data center to grow and change over time. A modular data center may consist of data center equipment contained within shipping containers or similar portable containers. But it can also be described as a design style in which components of the data center are prefabricated and standardized so that they can be constructed, moved or added to quickly as needs change.
The digital storage market doubles every 18 months, which translates to an annual growth rate of approximately 150% for the next 5 years. Standard brick-and-mortar data centers require significant capital expenditure (CAPEX) and operational expenditure (OPEX), which cause problems for businesses. However, modular data centers require approximately 50% lower construction costs in comparison to standard brick-and-mortar data centers. Also, modular data centers can be deployed within 12 to 15 weeks in comparison to 2 years needed by standard brick-and-mortar data centers. Existing businesses in need of additional data storage require highly efficient and reliable modular data centers to reduce the overall total cost of ownership (TCO).
Various companies including IO, HP, Dell, Google, and Colt supply modular data centers. However, these modular data centers are in some respects rudimentary with respect to energy efficiency. Moreover, the cooling mechanisms of these modular data centers are also rudimentary and inefficient with respect to energy efficiency. In addition, the electrical infrastructure is inefficient or does not have a redundant supply path. Most electrical infrastructure uses standard shipping containers for storage applications. Since it assumed in the art that efforts at improving efficiency will result in a decrease in reliability, comparatively little attention has been focused on improving the power efficiency of the data centers while at the same time maintaining the same or even greater levels of reliability.
With respect to the large conventional modular data centers, FIG. 1A shows an existing data center 10 with a centralized alternating current (AC) UPS 40, i.e., a double conversion AC-DC/DC-AC, and double conversion AC-DC/DC-DC server power supplies 45a-45n with an AC input and a DC output. Information technology (IT) loads 55 and mechanical loads 60 (e.g., the energy required by cooling systems) of the data center 10 are powered entirely by a utility feed 20 via an on-line double conversion AC-UPS 40 through a step-down transformer 35. A generator 15 starts to operate once a disturbance in the utility 20, e.g., a loss of all or a portion of the electricity provided by the utility feed 20, is more than approximately two seconds. During a disturbance in the utility feed 20, surge protector 25 dampens the disturbance. For disturbances beyond a pre-determined acceptable level that are beyond the dampening capabilities of the surge protector 25, the IT loads 55 and mechanical loads 60 are powered by the AC UPS 40 via one or more internal batteries and an internal DC-AC inverter section, neither of which are shown but are part of AC UPS 40.
Once the generator 15 has reached its reference speed and stabilized, the transfer switch 30 shifts the primary power source from the utility feed 20 to the generator 15. Thereafter, the IT loads 55 and mechanical loads 60 are entirely powered by the generator 15 via the on-line UPS 40. The internal batteries (not shown) of AC UPS 40 are also recharged by the generator 15. Once the disturbance in the utility feed 20 is no longer present, the IT loads 55 and mechanical loads 60 are shifted from the generator 15 to the UPS system 40. Ultimately, the transfer switch 30 shifts the primary power source from the UPS system 40 back to the utility feed 20.
A mechanical cooling system (not shown) is in thermal communication with each of a plurality of IT server racks 50a-50n, and circulates a coolant that removes heat generated by the plurality of IT server racks 50a-50n. The coolant is pumped by a cooling distribution unit (CDU) 65a-65n that includes a heat exchanger that allows the system to use refrigerant cooling. Each CDU 65a . . . 65n may support approximately 350 kW of IT load capacity, i.e., part of the mechanical load 60.
Turning now to FIGS. 2 and 3, there is illustrated a data center 100 having AC UPSs 145a, 145b and a server power supply with an AC input. During normal operation, the IT or server loads 50 (50a . . . 50n) and mechanical loads 65a-1 . . . 65n-1 and 65a-2 and 65n-1 of data center 100 (see FIG. 3) are powered entirely by the utility power feeders 105a, 105b (see FIG. 2) via on-line double conversion AC UPSs 145a, 145b that are similar to the centralized AC UPS 40 described above with respect to FIG. 1A. The utility power source is usually connected to the data center 100 through the first utility power feeder 105a. The second utility power feeder 105b is normally open at switch 105′ and supplies the data center load (both IT and mechanical loads 55, 60 as described above with respect to FIG. 1A) in case the first utility power feeder 105a malfunctions. The utility power feeders 105a and 105b supply power through switchgear 110 to feed step-down transformer 115. The voltage output of step-down transformer 115 is supplied to a first common bus 120. Power feed 122, in turn, supplies power from the first common bus 120 to a second common bus 135.
In turn, power feed 150 supplies power from the second common bus 135 to first primary common bus 150a via branch bus line 1501 and to second primary common bus 150b via branch bus line 1502. Power from first primary common bus 150a is supplied in turn to first secondary common bus 162a and also to second secondary common bus 162b via feeds 152a and 152b, respectively. Similarly, power from second primary common bus 150b is supplied in turn to first secondary common bus 162a and also to second secondary common bus 162b via feeds 154a and 154b, respectively.
Power is supplied to transformers 170a and 170c from first secondary common bus 162a via feed 164a and split feed 164a1 to transformer 170a and via split feed 164a2 to transformer 170c. Similarly, power is supplied to transformers 170b and 170d from second secondary common bus 162b via feed 164b and split feed 164b1 to transformer 170b and via split feed 164b2 to transformer 170d. Power is supplied directly from first secondary common bus 162a via feed 166a to transformer 170e and directly from secondary common bust 162b via feed 166b to transformer 170f. 
When a disturbance in the utility power feeders 105a, 105b occurs that is more than about two seconds, the generators 140a, 140b start. The disturbance is detected in bus 135 and generator 140a supplies power to feed 1351 through AC UPS 145a while generator 140b supplies power to feed 1352 through AC UPS 145b. AC UPS 145a then supplies power to bus 150a via branch bus feeder 13511 and to bus 150b via branch bus 13512. Similarly, AC UPS 145b then supplies power to bus 150a via branch bus feeder 13521 and to bus 150b via branch bus 13522.
In some cases, only one of generators 140a and 140b may start depending on the magnitude of the IT or server loads 50 (50a . . . 50n) and mechanical loads 65a-1 . . . 65n-1 and 65a-2 . . . 65n-1. When the generators 140a, 140b start, the IT and mechanical loads are still powered by the UPSs 145a, 145b via inverter 430 and battery 410 (see FIG. 4). When the generators 140a, 140b have reached their reference speeds and stabilized, the transfer switch (not shown) shifts the primary power source from the utility power feeder 105a to the generators 140a, 140b. Thereafter, the loads IT or server loads 50 (50a . . . 50n) and mechanical loads 65a-1 . . . 65n-1 and 65a-2 . . . 65n-2 are entirely powered by the generators 140a, 140b via the UPSs 145a, 145b. The UPS batteries 410 (See FIG. 4) are recharged by power generators 140a, 140b. When the disturbance in the utility power feeder 105a is no longer present, the loads 55 and 60 are shifted from the generators 140a, 140b to the UPSs 145a, 145b and ultimately transfer switch shifts the primary power source to the utility power feeder 105a. Transformer 115 steps down the voltage from the utility feed 105a. 
An auxiliary distribution source 160 and transformer 165, which are electrically coupled in series via feed bus 1601 and then via split feed bus 16011 with first primary common bus 150a and then via split feed bus 16012 with second primary common bus 150b, supply power to the loads IT or server loads 50 (50a . . . 50n) and mechanical loads 65a-1 . . . 65n-1 and 65a-2 . . . 65n-2 upon failure of the UPSs 145a, 145b and/or generators 140a, 140b. Wrap up lines 125a, 125b, which are electrically coupled to first common bus 120, provide an alternative path for supplying power to the loads if a problem arises in the AC UPSs 145a, 145b and/or generators 140a, 140b. Wrap up lines 125a and 125b include switches 130a and 130b, respectively, which are normally open and provide power if other lines electrically coupled to main switch gear 120 fail. Switch gears 162a and 162b allow for either of the UPSs 145a, 145b and/or either of the generators 140a, 140b to supply all or a part of the power for the entire IT or server loads 50 (50a . . . 50n) and mechanical loads 65a-1 . . . 65n-1 and 65a-2 . . . 65n-2 (see FIG. 1A).
Transformers 170c, 170d supply power to IT or server loads 50 (50a . . . 50n) via switches 185a and 185b, respectively. If either transformer 170c or 170d fails, then tie 187 assists in supplying power to bus 190a or 190b. Mechanical transformers 170a, 170b supply power to mechanical (CDU) loads 65a-1 . . . 65n-1 and 65a-2 . . . 65n-2 via switches 175a and 175b, respectively (see FIG. 1A). If either transformer 170a or 170b fails, then tie 177 assists in supplying power to bus 180a or 180b. Administration transformers 170e, 170f supply power to an administration building load (not shown) via switches 195a and 195b respectively. If either transformer 170e or 170f fails, then tie 197 assists in supplying power to the administration building load.
Either transformer 170a or 170b has sufficient capacity to handle the entire mechanical (CDU) loads 65a-1 . . . 65n-1 and 65a-2 . . . 65n-2 alone in case of failure of the other. However, transformers 170a and 170b generally work in combination, each carrying 50% of the load. Similarly, either transformer 170c or 170d has sufficient capacity to handle the entire IT or server loads 50 (50a . . . 50n) alone in case of failure of the other. However, transformers 170c and 170d generally work in combination, each carrying 50% of the load. Buses 190a and 190b are for IT or server loads 50 (50a . . . 50n) and buses 180a and 180b are for mechanical/CDU loads 65a-1 . . . 65n-1 and 65a-2 . . . 65n-2.
FIG. 3 shows connections between buses 180a, 180b and mechanical/CDU loads 65a-1 . . . 65n-1 and 65a-2 . . . 65n-2 and between buses 190a, 190b and IT load 50. More particularly, mechanical CDU loads 65a-1 . . . 65n-1 are supplied power from bus 180a and CDU loads 65a-2 . . . 65n-2 are supplied power from bus 180b. 
IT load 50 includes a plurality of server rack assemblies 50a . . . 50n, which are separated from each other to define hot aisles 210 and cold aisles 220. Each server rack assembly 50a . . . 50n is electrically coupled to buses 190a and 190b via server power supplies 500 (see FIG. 5). Each server power supply 500 includes an AC-DC converter 520a . . . 520n electrically coupled in series to a DC-DC converter 510a . . . 510n having outputs 530a, 530b, 530c, 530d electrically coupled to the respective IT or server loads 50a . . . 50n. 
FIG. 4 is a block diagram of an AC UPS 400 that can be used as the AC UPSs 145a, 145b of FIG. 2. The AC UPS 400 includes an AC-DC converter 440, a DC-DC converter 420, a battery 410, and a DC-AC inverter 430. As shown, the DC-DC converter 420 and the battery 410 are electrically coupled in parallel between the AC-DC converter 440 and the DC-AC inverter 430. The AC-DC converter 440 receives a high AC voltage via a plurality of power lines 460a . . . 460c and converts the high AC voltage to a high DC voltage. The high DC voltage is supplied to both the bidirectional DC-DC converter 420 and the DC-AC inverter 430. The DC-DC converter 420, which is a buck-boost converter, steps down the high DC voltage to a lower voltage that is suitable for charging lead acid battery 410 (e.g., an intermediate voltage).
In this case, high DC voltage is defined as about 1000 V and an intermediate voltage for the battery 410 is about 300 V DC to about 600 V DC. The three-phase inverter 430 also includes three outputs, that is, 3-phase 480 V AC outputs 450a, 450b, 450c. It should be noted that the individual 3-phase 480 V AC outputs 450a, 450b, 450c are not explicitly shown in FIGS. 2 and 3 but are represented as single phase lines in a single line diagram. The UPS efficiency in double-conversion mode is around 94%-96% at nominal load.
During normal operation, the DC-AC inverter 430 converts the high DC voltage from the AC-DC converter 440 into an AC voltage, which is supplied to the server power supplies 500 via step-down transformers 170c, 170d of FIG. 2. When there is a disturbance in the high AC voltage supplied by a utility to the AC-DC converter 440, the DC-DC converter 420 converts the voltage of the battery 410 into a high DC voltage, which is supplied to the DC-AC inverter 430. The AC UPS 400 is a double-conversion AC UPS because it performs two electrical conversions via the AC-DC converter 440 and the DC-AC inverter 430.
FIG. 5 shows server power supply 500, which includes two AC inputs (single phase) 540a, 540b from respective IT buses 190a, 190b (see FIG. 3). The AC-DC converter 520 converts a single phase AC voltage of the AC inputs 540a, 540b to an intermediate DC voltage. DC-DC converter 510 converts the intermediate DC voltage into multiple low DC voltages 530a . . . 530d. For example, the plurality of DC-DC converters 510a . . . 510n can supply approximately 3.3 VDC, 5 VDC, 12 VDC, and −12 VDC via phase output lines 530a, 530b, 530c, and 530d, respectively, to the respective servers 50a . . . 50n as shown in FIG. 3.
One disadvantage of the existing data center 100 is the double-conversion AC UPS 400. The two electrical conversions performed by the AC UPS 400 increase losses and increase power usage effectiveness (PUE) of the data center. PUE is a measure of how efficiently a data center uses its power. Specifically, PUE is a measure of how much of the power is actually used by the servers of the data center in contrast to the power used for cooling and other overhead functions of the data center. In other words, PUE is the ratio of the total amount of power used by a data center to the power delivered to the servers of the data center so that PUE is greater than 1.0, which is the ideal PUE value. Thus, the lower the PUE, the more efficient is the data center.
Another disadvantage of the existing data center 100 is the multiple electrical conversions (AC-DC/DC-DC) performed by the server power supply 500 (FIG. 5), which also increases losses and increases PUE. Therefore, the overall losses introduced by the AC UPS 400 and the server power supply 500 are high in the existing data center 100.
Alternatively, FIG. 1B shows a data center system 70 including a modular, scalable, double-conversion UPS system 72 and server power supplies with AC input voltages. More particularly, the server power supplies of the modular UPS system 72 include a plurality of in-line modular AC UPSs 75a . . . 75n that are each electrically coupled between the transformer 35 and a respective one of the plurality of server power supplies 45a . . . 45n. The modular UPS system 72 also includes a plurality of in-line AC UPSs 85a . . . 85n that are each electrically coupled between the transformer 35 and a respective one of the plurality of CDUs 65a . . . 65n. The efficiency of the modular UPS system 72 is high because of the higher loading factor of the modular UPS system 72 of FIG. 1B as compared to the centralized UPS system 40 of FIG. 1A, where the initial loading factor may not be high. Normally, the capacity of the centralized UPS system 40 of FIG. 1A is selected based on future load demand.
FIG. 6 illustrates an existing modular data center 1100 powered by a utility feed 1110 (e.g., 480 V L-L 3-phase AC utility feed). Multiple IT or server loads 1140a . . . 1140n are supplied by a 3-phase double conversion on-line UPS 1120. DC power is supplied to the IT loads 1140a . . . 1140n from, for example, 277 V L-N, 1-phase double conversion AC-DC/DC-DC converters 1130a . . . 1130n. The single phase (L-N) is supplied via the UPS 1210. The DC-DC output of the server power supplies or converters 1130a . . . 1130n is supplied to the IT loads 1140a . . . 1140n. A cooling mechanism 1115 (or mechanical load) is supplied by the utility feed 1110 (e.g., 480 V L-L 3-phase AC).
The cooling mechanism 1115 may employ Computer Room Air Conditioning (CRAC) cooling which is a central unit within the modular data center 100 that is relatively inefficient.
FIG. 7 illustrates another existing modular data center 1200 powered by utility feed 1110 (e.g., 480 V L-L 3-phase AC). The AC inputs of the server power supplies 1130a . . . 1130n are powered by a plurality of rack mountable 1-phase double conversion on-line AC UPSs 1120a . . . 1120n. In other words, each server power supply 1130a . . . 1130n is directly connected to an AC UPS 1120a . . . 1120n, in contrast to the system 1100 illustrated in FIG. 6. As such, every IT load 1140a . . . 1140n is powered by separate and distinct AC UPSs 1120a . . . 1120n. The system 1200 may also include a CRAC cooling mechanism 1115, as described above with reference to FIG. 6.
FIG. 8 illustrates a block diagram of 1-phase double conversion on-line AC UPSs 1120a . . . 1120n of FIG. 7. AC UPSs 1120a . . . 1120n each includes a 1-phase AC-DC converter 1310, a DC-DC converter 1320, a battery 1330, and a DC-AC 1-phase inverter 1340. As compared to the AC UPS 400 in FIG. 4, the AC UPS 400 is a 3-phase version of a double-conversion AC UPS whereas AC UPSs 1120a . . . 1120n are single phase versions of the double conversion AC UPS 400. The 3-phase AC UPS 400 has a larger power rating and cannot be mounted on the server racks 50a . . . 50n. However, 1-phase AC UPSs 1120a . . . 1120n may be mounted on the server rack 50a . . . 50n as they have a smaller power rating.
As shown in FIG. 8, the DC-DC converter 1320 and the battery 1330 are electrically coupled in parallel between the AC-DC converter 1310 and the DC-AC inverter 1340. The AC-DC converter 1310 receives a 1-phase medium AC voltage (e.g., 277 V AC) via a plurality of power lines 1405a, 1405b and converts the medium AC voltage to a medium DC voltage. The medium DC voltage is supplied to both the bidirectional DC-DC converter 1320 and the 1-phase DC-AC inverter 1340. The inverter 1340 also includes two outputs, that is, outputs 1445a, 1445b. The DC-DC converter 1320, which is a buck-boost converter, steps down the medium DC voltage to a lower voltage that is suitable for charging the lead acid battery (e.g., an intermediate voltage).
In view of foregoing, there are multiple conversions required for the supply of electrical power in conventional modular data centers and modular data pods.